Papers with generation pipeline
Instruction Tuning with Human Curriculum (2024.findings-naacl)
Copied to clipboard
| Challenge: | a recent study shows that human curriculum-inspired strategies can enhance performance of large language models. |
| Approach: | They propose a method for generating instruction-response datasets that emulate human learning . they find that substantial improvements can be achieved through curriculum ordering . |
| Outcome: | The proposed method achieves performance improvements on truthfulQA, MMLU, OpenbookQA, and ARC-hard benchmarks without additional computational costs. |
SuperWriter: Reflection-Driven Long-Form Generation with Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Long-form text generation remains a challenge for large language models . generating extended sequences often leads to degraded coherence and logical consistency . |
| Approach: | They propose a framework that integrates explicit structured thinking into long-form text generation. |
| Outcome: | The proposed framework surpasses even larger-scale models in evaluation and human evaluation. |
Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Text (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) can be used to effectively utilize tools in multi-turn interactions, but acquiring diverse and realistic multi-step tool-use data remains a challenge. |
| Approach: | They propose a text-based data synthesis pipeline that generates multi-turn tool-use trajectories from text corpora using relevance filtering, workflow tool extraction, trajectory grounding, and complexity refinement. |
| Outcome: | The proposed model achieves 14.9% improvement on the BFCL V3 Multi-turn benchmark while significantly reducing inference latency and costs. |
Unleashing the Power of Large Language Models in Zero-shot Relation Extraction via Self-Prompting (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for zero-shot Relation Extraction (RE) lack detailed, context-specific prompts for understanding various sentences and relations. |
| Approach: | They propose a framework that uses a three-stage diversity approach to prompt LLMs by generating multiple synthetic samples that encapsulate specific relations from scratch. |
| Outcome: | The proposed framework outperforms existing LLM-based zero-shot RE methods on benchmark datasets and shows that it produces high-quality synthetic data that enhances performance. |
Task-Adaptive Tokenization: Enhancing Long-Form Text Generation Efficacy in Mental Health and Beyond (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate text in mental health are limiting, but they are effective for many tasks. |
| Approach: | They propose a task-adaptive tokenizer that allows for the integration of task-specific tokens into the pre-trained model's tokenization step. |
| Outcome: | The proposed tokenization approach improves generation performance on psychological question-answering tasks in Chinese and English while using 60% fewer tokens. |
PRISM: Probing Reasoning, Instruction, and Source Memory in LLM Hallucinations (2026.acl-long)
Copied to clipboard
Yuhe Wu, Guangyu Wang, Yuran Chen, Jiatong Zhang, Yutong Zhang, Yujie Chen, Jiaming Shang, Guang Zhang, Zhuang Liu
| Challenge: | Existing benchmarks for hallucination evaluation rely on mixed queries and posterior evaluation, which quantifies hallucinosity severity but offers limited insight into where and why they occur. |
| Approach: | They propose a controlled benchmark that disentangles hallucinations into four dimensions: knowledge missing, knowledge errors, reasoning errors, and instruction-following errors. |
| Outcome: | The proposed model disentangles hallucinations into four dimensions: knowledge missing, knowledge errors, reasoning errors, and instruction-following errors. |